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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 

Listing of Claims: 

1. (Currently Amended) A hyperlinked broadcast system comprising: 

a video source providing video information for a video program including a plurality of 
video frames ; 

an annotation system generating annotation data to be associated with said video 
information and generating annotation data timing information , wherein the generating of the 
annotation data includes generating a single mask for each of the plurality of video frames, each 
mask including a plurality of pixels, wherein each of the plurality of pixels is mapped to an 
indicia for identifying a region or video object appearing in the corresponding video frame, 
wherein each pixel associated with the same region or video object is mapped to the same 
indicia, each mask further including graphics data for at least one of a plurality of video objects 
appearing in the corresponding video frame ; and 

an augmented video information transmission generator receiving said annotation data, 
said video information, and said annotation data timing information, said augmented video 
information transmission generator generating an augmented video transmission signal 
comprising said annotation data, said annotation data timing information, and said video 
information, 

wherein said augmented video information transmission generator associates said video 
information with said annotation data using said annotation data timing information. 

2. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises a vertical blanking interval insertion device. 



-2- 



Appln No. 09/943,583 

Amdt date June 8, 2006 

Reply to Office action of March 9, 2006 

3. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises at least one of a vertical ancillary data insertion device and a 
digital video data multiplexer. 

4. (Original) The system of claim 1 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 

5. (Original) The system of claim 1 wherein said video information comprises 
digital video data. 

6. (Original) The system of claim 1 wherein said video information comprises 
an analog video signal. 

7. (Original) The system of claim 1 further comprising: 
a post production environment; and 

a headend comprising said augmented video information transmission generator, 
wherein said video information and said annotation data timing information are combined 
by said post production environment and transmitted to said headend. 

8. (Original) The system of claim 7 wherein said headed is a cable headend. 

9. (Original) The system of claim 7 wherein said headend is a satellite headend. 
10-12. (Canceled) 
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13. (Original) The system of claim 1 further comprising: 

a receiver in communication with said augmented video information transmission 
generator; and 

a display device in communication with said receiver, wherein said receiver synchronizes 
said annotation data with said video information on a frame by frame basis. 

14. (Original) The system of claim 13 wherein said display device displays said 
annotation data in response to a viewer request. 

15-16. (Canceled) 

17. (Currently Amended) The system of claim [[15]] I wherein at least one of the 
[[said]] masks [[data]] comprises location information of an object in an annotated the 
corresponding video frame. 

18. (Original) The system of claim 17 wherein said location information includes 
a graphics location reference that represents a fixed relation to a set of pixels associated with said 
object. 

19. (Original) The system of claim 18 wherein said graphics location reference 
includes an upper left most pixel in said associated pixel set. 

20. (Original) The system of claim 18 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 

21. (Currently Amended) The system of claim [[15]] I wherein [[said]] at least one of 
the masks [[data]] comprises location and shape information of an object in [[a]] the 
corresponding video [[fame]] frame to be annotated . 
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22. (Currently Amended) A hyperlinked transmission assembly system comprising: 

a video information source providing video information for a video program including a 
plurality of video frames; 

an annotation data stream generator capable of accessing annotation data including a 
single mask for each of the plurality of video frames, each mask including a plurality of pixels, 
wherein each of the plurality of pixels is mapped to an indicia for identifying a region or video 
object appearing in the corresponding video frame, wherein each pixel associated with the same 
region or video object is mapped to the same indicia, each mask further including graphics data 
for at least one of a plurality of video objects appearing in the corresponding video frame ; 

a vid e o information source providing video information; 

an annotation data timing information decoder in communication with said annotation 
data stream generator and said video information source, said annotation data timing information 
decoder extracting annotation data timing information from said video information; and 

an augmented video information transmission generator in communication with said 
annotation data stream generator and said video information source, wherein said video 
information transmission generator synchronizes said video information with said annotation 
data based on said annotation data timing information. 

23. (Original) The system of claim 22 wherein said video information 
transmission generator synchronizes said video information with said annotation data on a frame 
by frame basis. 

24. (Original) The system of claim 22 wherein said annotation data timing 
information decoder is a vertical blanking interval decoder. 
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25. (Original) The system of claim 22 wherein said annotation data timing 
information decoder is at least one of a vertical ancillary data decoder and a digital transport 
stream decoder. 

26. (Currently Amended) The system of claim 22 wherein said annotation data 
stream generator storage devic e is capable of accessing said annotation data at least as early as 
said annotation data stream generator receives said annotation data timing information. 

27. (Original) The system of claim 22 wherein said annotation data stream 
generator accesses said annotation data from an internal storage device. 

28. (Original) The system of claim 22 wherein said annotation data stream 
generator accesses said annotation data from an external storage device. 

29. (Original) The system of claim 22 wherein said annotation data stream 
generator streams said annotation data in response to said annotation data timing information. 

30. (Original) They system of claim 22 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and 
video program identifier. 

31. (Canceled) 

32. (Currently Amended) A hyperlinked reception system comprising: 

a receiver in communication with a broadcast chann e l receiving a transmission signal 
encoded with annotation data, the annotation data including a single mask for each of a plurality 
of video frames of a video program, each mask including a plurality of pixels, wherein each of 
the plurality of pixels is mapped to an indicia for identifying a region or video object appearing 
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in the corresponding video frame, wherein each pixel associated with the same region or video 
object is mapped to the same indicia, each mask data packet further including graphics data for at 
least one of a plurality of video objects appearing in the video frame; 

a video decoder extracting the annotation data from the transmission signal ; and 
a display device; in communication with said receiver, wher e in said receiv e r 
a memory storing the extracted annotated data; and 

a central processing unit in communication with said memory, the central processing unit 
being configured to synchronizers]] the masks [[data]] with associated video information on a 
frame by frame basis in response to timing information. 

33. (Currently Amended) The system of claim 32 wherein said receiver comprises a 
timer to calculate an offset from said timing information to synchronize said masks [[data]] with 
said associated video information. 

34. (Original) The system of claim 32 wherein said timing information comprises 
at least one of timestamp information, timecode information, frame numbering information, 
global time of day information, receiver commands, and video program identifier. 

35. (Currently Amended) A method of generating a hyperlinked video signal 
comprising: 

generating annotation data timing information from video information for a video 
program including a plurality of video frames ; 

generating annotation data for said video information , wherein the generating of the 
annotation data includes generating a single mask for each of the plurality of video frames,, each 
mask including a plurality of pixels, wherein each of the plurality of pixels is mapped to an 
indicia for identifying a region or video object appearing in the corresponding video frame, 
wherein each pixel associated with the same region or video object is mapped to the same 
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indicia, each mask further including graphics data for at least one of a plurality of video objects 
appearing in the corresponding video frame ; 

communicating said annotation data timing information, said annotation data, and said 
video information to an augmented video information transmission generator; and 

synchronizing said video information with said annotation data in response to said 
annotation data timing information by said augmented video information transmission generator. 

36. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises a vertical blanking interval insertion device. 

37. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises at least are of a vertical ancillary data insertion 
device and a digital video data multiplexer. 

38. (Original) The method of claim 35 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 

39. (Original) The method of claim 35 wherein said video information comprises 
digital video data. 

40. (Original) The method of claim 35 wherein said video information comprises 
an analog video signal. 

41. (Original) The method of claim 35 further comprising inserting said 
annotation data timing information in a vertical blanking interval of an analog video signal. 
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42. (Original) The method claim 35 further comprising inserting said annotation 
data timing information in a vertical ancillary data region of a digital video signal. 

43. (Original) The method of claim 35 wherein said communicating step 
comprises transmitting said timing information and said video information to a broadcast 
network and subsequently to said augmented video information transmission generator. 

44-45. (Canceled) 

46. (Currently Amended) The method of claim [[44]] 35 wherein [[said]] at least one 
of the masks [[data]] comprises location information of an object in the corresponding an 
annotated video frame. 

47. (Original) The method of claim 46 wherein said location information includes 
a graphics location reference that represents a fixed relation to a set of pixels associated with said 
object. 

48. (Original) The method of claim 47 wherein said graphics location reference 
includes an upper left most pixel of said associated pixel set. 

49. (Original) The method of claim 48 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 

50. (Currently Amended) The method of claim 44 wherein [[said]] at least one of the 
masks [[data]] comprises location and shape information of an object in an annotat e d the 
corresponding video frame. 
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51. (Original) The method of claim 50 wherein said shape information is 
represented by a graphical overlay of said object. 

52. (Original) The method of claim 50 wherein said shape information is 
represented by an outline of said object. 

53. (New) The system of claim 1, wherein each mask includes graphics data for a 
plurality of video objects appearing in the corresponding video frame. 

54. (New) The system of claim 1, wherein each mask further includes timing 
information of the corresponding video frame, wherein the augmented video information 
transmission generator associates the video information with the annotation data based on a 
comparison of the annotation data timing information and the timing information included in the 
masks. 

55. (New) The system of claim 1, wherein the annotation data further includes a 
plurality of object data packets, the mask further including an identifier to an object mapping 
table included in one or more of the plurality of object data packets, the object mapping table 
including at least one entry with an indicia from the corresponding mask identifying a particular 
video object, the entry further associating the indicia to information data structures included in 
one or more of the plurality of object data packets, the information data structures including 
information for the particular video object. 

56. (New) The system of claim 55 further including a television receiver, the 
television receiver being configured to: 

overlay a graphics image on a particular video frame for the particular video object based 
on the graphics data included in the corresponding mask; 
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retrieve the identifier of the object mapping table from the corresponding mask 
responsive to a user selection associated with the overlaid graphics image; 
retrieve the object mapping table based on the retrieved identifier; 

identify the indicia in the corresponding mask for the particular video object for which 
the graphics image was overlaid; 

locate the entry in the object mapping table with the identified indicia; 
identify the information data structures associated with the located entry; 
retrieve the information in the identified information data structures; and 
display the retrieved information on the display device. 

57. (New) The system of claim 32, wherein each mask includes graphics data for a 
plurality of video objects appearing in the corresponding video frame. 

58. (New) The system of claim 32, wherein each mask further includes the timing 
information of the corresponding video frame, wherein the central processing unit synchronizes 
the plurality of masks with the video information based on the timing information included in the 
masks. 

59. (New) The system of claim 32, wherein the annotation data further includes a 
plurality of object data packets, the mask further including an identifier to an object mapping 
table included in one or more of the plurality of object data packets, the object mapping table 
including at least one entry with an indicia from the corresponding mask identifying a particular 
video object, the entry further associating the indicia to information data structures included in 
one or more of the plurality of object data packets, the information data structures including 
information for the particular video object. 
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60. (New) The system of claim 59, wherein the central processing unit is further 
configured to: 

overlay a graphics image on a particular video frame for the particular video object based 
on the graphics data included in the corresponding mask; 

retrieve the identifier of the object mapping table from the corresponding mask 
responsive to a user selection associated with the overlaid graphics image; 

retrieve the object mapping table based on the retrieved identifier; 

identify the indicia in the corresponding mask for the particular video object for which 
the graphics image was overlaid; 

locate the entry in the object mapping table with the identified indicia; 
identify the information data structures associated with the located entry; 
retrieve the information in the identified information data structures; and 
display the retrieved information on the display device. 
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